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Abstract 

This paper proposes iterative Bayesian method to estimate success probability based 
on unique sample. The procedure is replacing the distribution characteristic of prior 
with Bayes estimate on the every iteration until they coincide. Iterative Bayes estimate 
is generally independent of hyperparameters. For binomial, Poisson, exponential and 
normal model, iterative limit is shown to be MLE in case the expectation of conjugate 
prior is replaced respectively. Particularly, suppose success appears in one and only trial, 
while the mode of triangle prior is replaced iterative Bayesian method gives 1/0 w 0.618 
{4> is Golden Ratio) as the estimate of success probability p, this result reveals the truth 
of Golden Ratio from the point of statistics. Furthermore, under triangle prior the 
unique sample X from binomial model B(n,p) is considered. Existence and uniqueness 
of iterative Bayes estimator pi b for parameter p is given. 
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1. Introduction 

The classical estimator for success probability p is frequentist estimator p n = ^ 
where X ~ B(n,p). This estimator has many nice properties including that it is MLE 
and UMVUE (Lehmann and Casella, 1998). It is inevitable that or 1 will be given as 
the estimate of p when the sample size n is small, especially n = 1 . 
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Another useful estimator is based on Bayesian method. Suppose a prior distribution 
of p is Beta distribution Beta(pt, (3) which is conjugate prior: 

*W = fT ! wm^~ 1 0--p) p ~ 1 ' < p <!;<*> 0,0 >o- (i) 

The Bayes estimator of p for the quadratic loss function is the expectation of posterior 
distribution: p = a "^ n , here the unknown hyperparameters a and /3 should be given 
by past knowledge or estimated from the samples. Uniform prior (a = j3 = 1) was a 
favorite of Laplace and Bayes, now Jeffreys prior (a = = ^) is popular (Jeffreys, 1946; 
Ghosh ct al., 2006). 

We consider the estimation of success probability based on one and only sample in 
this paper. Such unique sample problems arise in many applications, in particular in 
the fields of nature and modern high-tech. For the events in nature limited by time, or 
the extremely expensive trials in high-tech, we usually encounter the difficulty that there 
is unique sample available because the trial is unrepeatable. This naturally leads to an 
interesting question as follows: 

When only one trial is made or there is unique sample is available, meanwhile there 
is no any past knowledge to use, what is the most reasonable estimate of the probability 
p of an event? 

Many excellent statisticians have noticed this problem. Efron (1973) developed Boot- 
strap resampling method for small sample sizes problem. It's unfortunate that Bootstrap 
is ineffective in case of sample size n = 1. Wasserman (2004) concluded that in prac- 
tice statistical methods based on very small sample sizes might not be reliable. Even 
in the classical binomial model, as Lehmann and Casella (1998) (pp. 232) pointed out: 
"in practice it is rare for either of the two assumptions ((i) independence of the n trials 
and (ii) constancy of the success probabilities p throughout the series) to hold exactly 
- consecutive trials typically exhibit some dependence and success probabilities tend to 
change over time." So it is completely necessary to research point estimation based on 
unique sample, especially estimate success probability in case only one trial is made. 

The outline of this paper is as follows. Section 2 introduces iterative Bayesian method. 
Section 3 and Section 4 estimate success probability under triangle prior based on unique 
sample or very small sample sizes respectively. In Section 2, for binomial model we re- 
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place the distribution characteristic of conjugate Beta(a, /?) prior with Bayes estimate 
until they coincide, the iterative limit is shown to be MLE — replaced the expectation of 
prior, and that is the uniform prior Bayes estimator replaced the extreme point of 
prior respectively. It is remarkable that iterative Bayes estimate is independent of hyper- 
parameters, and for Poisson model, exponential model and normal model, the iterative 
limit is also MLE replaced the expectation of conjugate prior. 

In Section 3, we consider that there is one and only trial and success appears, by using 
our iterative method to replace the mode of a triangle prior, the surprising estimate of 
success probability is derived to be Y/<j> ~ 0.618, 4> is Golden Ratio. This result shows </> is 
the expected number of attempts. In Section 4, iterative Bayes estimator under triangle 
prior based on unique sample from binomial model has been investigated, its value isn't 
materially different from uniform prior Bayes estimator, whereas it is closer towards 0.5. 
Concluding remarks are given in Section 5. Main proofs are given in Appendix. 



2. Iterative Bayesian approach 

For binomial model B(n,p) we consider the Bayes estimator p = a "^ n under con- 
jugate prior Beta(a, /3) with unknown hypcrparametcrs a and /3. 

Firstly, now that -^h$ is the expectation of prior distribution, Bayes estimate as the 
expectation of posterior may be not far to it. We could hope to find better estimate of p 
if the expectation of prior is modified to be a "^ w . However there are two parameters 
in Beta{a, /?) prior, the density function of Beta(a, ft) isn't able to be confirmed by its 
expectation. 

Here we assume /3 is known as (3q (in fact /? is shown to be a nuisance parameter 
from the following iterative result). It's clear after m iterations the parameter a m is 
determined by 

a m +po a m -i+Po + n 

Then 

m— 1 a 

a m = (a + x)c m + x V c\ c= 

f—t Po + n-x 
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which yields after m iterations the Bayes estimate a a +p +n of p is 

a+(m+l)n 

Q+/3 +(m+l)n' X — U 

a(l — c)c m +x{l — c m ) 
a(l-c)e m -:EC>™+n ' X < n 

It is easily verified that the iterative limit corresponds to MLE ^ when m tends to 
infinite. 

Next, we shall turn to consider replacing the extreme point of prior. It can be 
shown that tt(p) in (1) attains its maximum at a °^l 2 ( a ^ ^ 01 mm imum at 
( Q < ^->P ^ ■*■)■ using the same iterative method as described above, we note 
that the iterative limit corresponds to uniform prior Bayes estimator if we replace 
the extreme point of prior with Bayes estimate on the every iteration. 

In general, for fixed < a < b, if characteristic a "^° b of prior is modified to be Bayes 
estimate a "^ n by assuming (3 is known as /3q, after m iterations we will find the Bayes 
estimate a Ta~ x , of p to be 



a.+n-\-m(a+n) 
a+7i+/3o+?n(a+?i) ' 



{a+x)(l-c)c m + (a+x)(l-c m ) 
(a+a;)(l — c)c m — {a+x)c mJ rn J rb ' 



a = b, x = n 



otherwize 



here c = ^ / b ^ . Therefore we have 

Po+n—x 

Theorem 1. Let X ~ B(n,p), the prior distribution of parameter p is Beta(a, fi), 
b (0 < a < 6, < b < 1) is a distribution characteristic of prior. If we replace 
this distribution characteristic of prior with the expectation of posterior on the every 
iteration, the iterative limit will exist and correspond to the estimator . 

As in the binomial case, it should be pointed out for Poisson model, exponential 
model and normal model, the iterative limit will correspond to MLE if the expectation 
of conjugate prior replaced with the expectation of posterior on the every iteration. See 
the following Table 1. 
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Table 1: Iterative Bayes estimate is equivalent to MLE 



population 


parameter 


conjugate 
prior 


expectation 
of prior 


expectation 
of posterior 


iterative limit 
{MLE ) 




A, A > 




£ 


a + n 




l e -*/A 
A L 


A, A > 






0+n-l 




N{h,<tI) 


/Lt, fj. E i? 


~ jV(a, /3 2 ) 




"n + ™^ 






e,e> o 


-s£_9i 8 - 1 e- 0,e 




2/3 + n 




r((S)° e 




2a+S(*i-fo) 2 





3. Golden Ratio estimate 

Assume the trial result is success when only one trial is made, and there is no prior 
knowledge of success probability p, then MLE gives 1 as the estimate for p. As a modifi- 
cation the Bayes estimate of uniform prior for p is | ~ 0.667 meanwhile that of Jeffreys 
prior is 0.75. Obviously it is not easy to choose one from | and 0.75 to be the estimate 
of success probability. 

Let us make choice of a triangle prior of success probability p as follows: 

!— p for < p < a 

a P J P- (2) 

j^(l-p) for a<p<l 

here hyperparamctcr < a < 1 is unknown. This triangle prior is natural because of 
unimodal with mode a, its shape is similar to that of Beta(a, /3) when a > > 1. 
As opposed to Jeffreys prior which implies success probability p is close to either or 1, 
triangle prior illustrates p is likely to be close to an unknown a. 

Now the joint density of X and p is 

—p 2 for < p < a 

T=^P(1 ~P) for a<p<\ 
which implies the marginal distribution of X is 

ra 2 , . f 1 2 , . 1 + a 



f(x,p) = 



f a 2 [ 2 

/ -p 2 dp+ p(l -p)dp 

Jo a J a 1 - a 
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Then the conditional density of p given x is 



6 m 2 



»(1 + Q) 
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p for < p < a 



T=^P0--P) f or ol<p<1 
which yields the Bayes estimate of p for the quadratic loss function is 

P= n , , P 3 dp+ / -p 2 {l-p)dp= 3 

Jo a(l + a) J a I -a 1 2(1 + a) 

By using an argument similar to the one applied in Section 2, we replace the mode 
a of triangle prior with Bayes estimate (3) on the every iteration (here triangle prior is 
completely decided on its mode). Hence the iterative limit r satisfies 



2(1 + r) 

which implies the iterative Bayes estimate of p is 

VE- 1 l 

P = — = 0.618. (4) 

This is an amazing result, the Golden Ratio (j> is considered to be the world's most 
astonishing number, not only appears in art and science, but also in natural structures 
(For details, see (Livio, 2002; Olscn, 2006)). There is no reason why we don't use ^ as 
the estimate of success probability when success appears in one and only trial. 



4. Unique sample from binomial model 

In this section we consider iterative Bayes estimator for p under triangle prior based 
on unique sample X from binomial model B(n,p) with n known. 



From (2), the joint density function of X and p is given by 
f(x,p) = 



I0p x+1 {l-p) n " x for0<p<a 
^("^(l-p)™-^ 1 fora<p<l 



which implies the Bayes estimate of p for the quadratic loss function is 

. I lo P X+2 ( l - P) rM d P+lh la P X + 1 ( l ~ pY M+1 d P 

I JoV +1 (l - P) n ~ x dp+j^ $ l a P x {l - p)"-^ 1 dp ' 



(5) 
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By adopting a method above used in deriving the Golden Ratio estimate, we replace the 
mode a of triangle prior with Bayes estimate (5) on the every iteration, therefore the 
iterative limit r satisfies 

|/ T (p x+2 (l-p) n - x - rp x+1 {l -p) n ~ x )dp 
= j^p J^{t P x (1 ~ p) n ~ x+1 - p x+1 (l ~ p) n - x+1 ) dp. 

Consider the change of variable: p — rt, this equation reduces to be 

T X+2 f X+1 (l - t)(l - Tt) n - X dt 

= (1 - T ) n - X + 2 J X t n ~ x+1 (l - t)(l - (1 - T)t) X dt. (6) 

Denote the function I n (a,x) of variable a on interval (0, 1) by 

/«(<*, a) =a x+2 f t x+1 (l-t)(l-at) n - x dt, 
Jo 

then the equation (6) can be rewritten as 

I n (r,x) = I n (l -r,n- x) 

which is natural because the estimator for p with observation value x is equivalent to 
that for 1 — p with observation value n — x. 

Noticing that for < a < 1, 



dl n (a, x) sr^ ( n ~ x \, , \n-x-r a 



7 n— r+1 



5a ^—^ \ r J (n — r + 3) 

= a* +1 / P +2 (l - at)™" 1 rft > 0, 



and 



o 



l n (0,x) = 0, I n (l,x) = , , o 



(«+3)c:?)' 

Hence I n (a,x) is strictly increasing from to hn+2\ on interval (0, 1). Similarly, 

I n (l — a,n—x) is shown to be strictly decreasing function from to on interval 

(0, 1). Therefore we have the following 
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Theorem 2. Let X ~ B(n,p), the triangle prior distribution of p is defined by (2), 
if we replace the mode a of prior with expectation of posterior on the every iteration, 
then the iterative limit (iterative Bayes estimate) exists and is unique. 

For convenience the iterative Bayes estimator given above in Theorem 2 is written to 
be pib- The following Table 2 presents some values of pis (n < 10) evaluated by Maple. 



Tabic 2: Iterative Bayes estimates in binomial model 



n \ x 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


0.382 


0.618 




















2 


0.309 


0.5 


0.691 


















3 


0.259 


0.419 


0.581 


0.741 
















4 


0.223 


0.361 


0.5 


0.639 


0.777 














5 


0.195 


0.317 


0.439 


0.561 


0.683 


0.805 












6 


0.174 


0.282 


0.391 


0.5 


0.609 


0.718 


0.826 










7 


0.157 


0.254 


0.352 


0.451 


0.549 


0.648 


0.746 


0.843 








8 


0.143 


0.231 


0.321 


0.410 


0.5 


0.590 


0.679 


0.769 


0.857 






9 


0.131 


0.212 


0.294 


0.377 


0.459 


0.541 


0.623 


0.706 


0.788 


0.869 




10 


0.121 


0.196 


0.272 


0.348 


0.424 


0.5 


0.576 


0.652 


0.728 


0.804 


0.879 



The symmetric structure of pis illustrated in Tabic 2 follows from the equation (6). 
In fact if we write the iterative Bayes estimate of p to be pib(x) with sample observation 
value x, pib(ji — x) means the iterative Bayes estimate of p with sample observation 
value n — x, it's clear that 

Pib(x) +PiB{n - x) = 1. (7) 

Furthermore, the following Theorem 3 gives the range of Pib(x) to be (f^p f+§)i 

which implies there is little difference between Iterative Bayes estimator pis and uniform 

prior Bayes estimator whereas pjb is closer towards 0.5. 

Theorem 3. (1). pjb(x) is the unique real zero lied in interval (0,1) of the 
polynomial with integer coefficients J n (a,x) denoted by 

n—x „ 

2a x + 2 V ( n+ )( X + r )(-l) r a r -{n-x + l){n + -i)a + {n-x + l){x + l). 
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(2). |±§ < Pib(x) < for x = 0, 1, • • • , n. More precisely, 

f £§<Pz B (*)<£|. for *<f ^ 
1 gHfrufr) < /or x> f. 

The proof of Theorem 3 is in Appendix. 

Remark. Unique sample from negative binomial model 

Suppose X is unique sample from negative binomial model NB{r,p) with r known, 
from the distribution of X 

p(x = x) = ( x + r x ~ i y*(i- P y, x = 0,1,2,-.., 

it is clear under triangle prior the iterative Baycs estimate of success probability p with 
sample observation x from NB(r,p) is equivalent to that with sample observation x from 
B(x + r,p). 

For geometric model G(p), X is the number of successes until the appearance of the 
first failure. Theorem 3 implies that the iterative Bayes estimate of p under triangle prior 
is the unique real zero lied in interval (0, 1) of the following polynomial 

(x + l)a x+3 -(x + 4)a x+2 + {x + A)a - (x + 1), x > 1. (9) 

From Table 2 we have the following 



Table 3: Iterative Bayes estimates in geometric model 



X 





1 


2 


3 


4 


5 


6 


7 


8 


9 


p 


0.382 


0.5 


0.581 


0.639 


0.683 


0.718 


0.746 


0.769 


0.788 


0.804 



5. Discussion 

We proposed iterative Bayesian method based on very small sample sizes, in particular 
unique sample. The idea is replacing a suitable distribution characteristic of prior with 
Bayes estimate on the every iteration until they coincide. Iterative Bayes estimate is a 
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limit of Bayes estimators, it can also be MLE for some models. We note that iterative 
Bayes estimate doesn't always exist. For example, iterative limit will be complex when 
the expectation of triangle prior rather than the mode is replaced in Section 3. It is 
interesting that under triangle prior iterative Bayes estimate of success probability is 
irrational (except for 0.5) in view of the parameter space (0, 1). 

There is a connection between our result and quantum theory. Quantum mechanics 
believes the physical universe is itself probabilistic rather than deterministic. Therefore 
our world consists of series of successes, each trial is unrepeatable. Golden Ratio estimate 
of success probability indicates the expected number of attempts is </>, maybe this is the 
reason Golden Ratio appears extensively in natural structures. 

We believe iterative Bayes estimate under triangle prior is reasonable for parameter 
estimation in unrepeatable trial (or based on unique sample). We are also interested 
in the simulations of some structures in nature such as spiral based on Golden Ratio 
estimate of success probability. 

Appendix Proof of Theorem 3 

As seen from the above arguments in Section 4, it is sufficient to express 



J n (a,x) = — --(1 - o)[7„(l - a,n-x) - I n (a,x) 

xlin — xv. 




Let us expand (1 — a)I n (l — a,n — x) as the following polynomial 



c + c x a + c 2 a 2 + • • • + c n+3 a 



n+3 



Firstly, 



(1 — a)J n (l — a,n — x 



r=0 ^ ' 



(n — r + 2)(n - r + 3)' 



(1 - a) n - r + 3 



By using the following combinatorial identity (Gould (1972), pp. 6, (1.41)) 




(10) 



it is clear that 



co = 



(n — x+ l)l(x + 1)! 
(n + 3)! 



ci = - 



{n — x + l)\x\ 



(n + 2)! 
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Next, 



[(1 - a) J„(l - a, n - *)] = £ ( J (-l)-'(l - a) n ~ r+1 



n—x-\-l—r^n—r-\-l 



r=0 
n -x + 1 , _ 

= (i - a ) n -+v = ^ r )(-l) 

which yields that (1 — a)/„(l — a, n — x) is equal to 

(n- a: + l)!(a; + l)! {n-x + l)\x\ ^ (n - x + 1\ . 1 , fc _ a a fe 



(n + 3)! (n + 2)! fc £^ 2 ~ x ~~ 2 ^ (fc-l)fc' 

On the other hand, 

^— ? Ai - t\ n n-r+2 _ n n-r+3 

(l-a)I n (a,x) = J2( )(-l) n — r 



(n - r + 2)(n-r + 3) 
.k-x-2) k(k + l) + 2-< \k-x-3J (fc-l)fc 



Hence, (1 — a)[/ n (l — a, n — cc) — x)] is derived to be 

(n-x + l)\(x + l)\ (n-x + l)\x\_ ^ 2 2(-l)*— »( fc »;f 2 ) fc 



(n + 3)! (n + 2)! ^ (fc - l)fc(fc + 1) 
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Simplifying and from the fact that 7„(1 — a, n — x) and / n (a, x) intersect in only one 
point in interval (0, 1), we have shown Pib(x) is the unique real zero lied in interval (0, 1) 
of J n (a,x). 

2. Recalling the relation of (7), to complete the proof of Theorem 3 we need only 
show 

x + 1 

— — <p IB (x), for x = 0,l,---,n; (11) 
n + 3 

and 



Pib{x) < ——, for x > -. (12) 
n + 2 I 



Consider the polynomial 



n—x 



H(a) = £( " + 3 )( X + r )(-l)W 
* — ' \n — x — r I \ r I 



r=0 

n — x 

E 

r=0 



n + 3 \/-( x +l) 
n — x — r ) \ r 
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which shows H(a) is the coefficient of the term t n x in the polynomial (1 + t) n+3 (l + 
at)~( x+1 > , denoted by 

H(a) = [t n ~ x ](l + t)" +3 (l + at)-( x+1 \ 

For < a < 1, 

H(a) = [t n - x }(l + at+(l-a)t) n+3 {l + at)-<- x+1) 

= [t n ~ X ] if ( U t 3 )^(1 - a) k (l + at) n - x - k+2 



fe=0 

n—x 



Ein + 3\fn~x~k + 2\ „___».,, >t. 
;0 ( fc J( 2 J a fe (l-«) fe >0 

which yields J n {f^,x) > from the fact 

J n (a, x) = 2a x+2 H(a) - (n - x + l)(n + 3)ct + (n - x + l)(x + 1). (13) 

Therefore, 

r 35+1 . X + 1 

I n {l —,n-x) > I n { — 

n + 3 n + 3 

The result (11) is true because I n (l — a, n — x) is strictly decreasing and I n (a, n — x) 
is strictly increasing in interval (0, 1), Pib(x) is their only one intersecting point in (0, 1). 

Wc finally prove (12). Similarly, it is enough to prove 

t I X + 1 1 n £ n 

•Jn( -x,x) < for X > — . 

n + 2 ; 2 

From (13), we need only show that 

'n + 3\ (n — x + 2 — k) (n — x + 1 — k) ,n — x + 1 



E 

k=0 



ti- 



ll — x + 1 x + 1 



< (1 + n ^|V +1 , n > 1, \ < x < n. (14) 
It can be seen when x = n, n — 1, n — 2, ■ • -, inequality (14) is 

2 <(i + -L T r+ 1 , 

3 + 2^<(l + ^)»+ 1 , 
n n 



i!i± 3 6 („ + 3 )(n + 2) 1 1+ 3 

n — 1 2 (n — 1)^ n — 1 
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Noticing both the coefficients of constant term in expansions of two side of (14) are 
equal to be: 

^ (n - x + 2 - k){n -x + l-k) t _ k _ x _ ( n - x + l) k 

k \ [n x + l) - ^ k\ ' 

k=0 k=0 

and the inequality (14) will become equality if n = 2x by using another Gould's identity 
(Gould (1972), pp. 11, (1.83)) 




Therefore (14) is obtained. 
References 

Efron, B., 1973. Bootstrap methods: Another look at the jackknifc. Annals Statist. 7, 
1-26. 

Ghosh, J.K., Delampady, M., and Samanta, T., 2006. An introduction to Bayesian anal- 
ysis - theory and methods. Springer, New York. 

Gould, H. W., 1972. Combinatorial identities. Morgantown, WV. 

Jeffreys, H., 1946. An Invariant form for the prior probability in estimation problems. 
Proc. Roy. Soc.(Ser. A). 186, 453-461. 

Lchmann, E. L. and Casella, G., 1998. Theory of point cstiamation.(2nd Ed). Springcr- 
Verlag, New York. 

Livio, M., 2002. The Golden Ratio: The story of Phi, the world's most astonishing 
number. Broadway Books, New York. 

Olsen, S., 2006. The Golden Section: Nature's greatest secret. Wooden Books (Walker 
& Company), New York. 

Wasserman, L., 2004. All of Statistics: a concise course in statistical inference. Springcr- 
Verlag, New York. 



13 



